Regularized Discriminant Analysis
نویسنده
چکیده
Linear and quadratic discriminant analysis are considered in the small sample high-dimensional setting. Alternatives to the usual maximum likelihood (plug-in) estimates for the covariance matrices are proposed. These alternatives are characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk. Computationally fast implementations are presented, and the efficacy of the approach is examined through simulation studies and application to data. These studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved. Submitted to Journal of the American Statistical Association *Work supported by the Department of Energy, contract DE-AC03-76SF00515.
منابع مشابه
Algorithms for Regularized Linear Discriminant Analysis
This paper is focused on regularized versions of classification analysis and their computation for highdimensional data. A variety of regularized classification methods has been proposed and we critically discuss their computational aspects. We formulate several new algorithms for regularized linear discriminant analysis, which exploits a regularized covariance matrix estimator towards a regula...
متن کاملRegularized Discriminant Analysis, Ridge Regression and Beyond
Fisher linear discriminant analysis (FDA) and its kernel extension—kernel discriminant analysis (KDA)—are well known methods that consider dimensionality reduction and classification jointly. While widely deployed in practical problems, there are still unresolved issues surrounding their efficient implementation and their relationship with least mean squares procedures. In this paper we address...
متن کاملRegularized Orthogonal Local Fisher Discriminant Analysis
Aiming at deficiencies of the ability for preserving local nonlinear structure of recently proposed Regularized Orthogonal Linear Discriminant Analysis (ROLDA) for dimensionality reduction, a kind of dimensionality reduction algorithm named Regularized Orthogonal Local Fisher Discriminant Analysis (ROLFDA) is proposed in the paper, which is originated from ROLDA. The algorithm introduce the ide...
متن کاملComputation of Regularized Linear Discriminant Analysis
This paper is focused on regularized versions of classification analysis and their computation for high-dimensional data. A variety of regularized classification methods has been proposed and we critically discuss their computational aspects. We formulate several new algorithms for shrinkage linear discriminant analysis, which exploits a shrinkage covariance matrix estimator towards a regular t...
متن کاملA Spatial Regularization of LDA for Face Recognition
This paper proposes a new spatial regularization of Fisher linear discriminant analysis (LDA) to reduce the overfitting due to small size sample (SSS) problem in face recognition. Many regularized LDAs have been proposed to alleviate the overfitting by regularizing an estimate of the within-class scatter matrix. Spatial regularization methods have been suggested that make the discriminant vecto...
متن کاملA Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis
Fisher linear discriminant analysis (LDA) and its kernel extension— kernel discriminant analysis (KDA)—are well known methods that consider dimensionality reduction and classification jointly. While widely deployed in practical problems, there are still unresolved issues surrounding their efficient implementation and their relationship with least mean squared error procedures. In this paper we ...
متن کامل